Operations Monitoring in an Area

ABSTRACT

An assembly for monitoring an area is provided. The assembly can include two or more cameras sensitive to radiation of distinct wavelength ranges. The fields of view of the cameras can be substantially co-registered at the area to be monitored. The assembly can include a computer system which can process the image data to monitor the area. The computer system can be configured to identify relevant objects present in the area, update tracking information for the relevant objects, and evaluate whether an alert condition is present using the tracking information.

REFERENCE TO RELATED APPLICATIONS

The current application is a continuation of U.S. patent applicationSer. No. 14/279,385, filed on 16 May 2014, which claims the benefit ofU.S. Provisional Application No. 61/855,517, filed on 17 May 2013, eachof which is hereby incorporated by reference.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under contract no.DTRT57-11-C-10041 awarded by the U.S. Department of Transportation. Thegovernment has certain rights in the invention.

TECHNICAL FIELD

The disclosure relates generally to monitoring operations in an area,and more particularly, to monitoring an area which includes a restrictedarea.

BACKGROUND ART

Current approaches for monitoring safe operations and providing warningsof unsafe circumstances in an area, such as an at-grade railroadcrossing, where different types of vehicles may operate generally fallinto two categories. The first category attempts to exclude vehicles andpersons from the grade crossing when a train is or may soon be passingthrough the crossing. These include simple “crossbuck” signage warningpedestrians and motorists that they are approaching an at-grade railroadcrossing, warning lights and bells, and large gates which lower at theapproach of a train and raise only after the train has passed.

These approaches are at best only partially successful. One problem isfailure of the mechanical and/or electrical systems on occasion, whichresults in the system giving no warning or activating far too late forany human being to react properly in time. Additionally, human operatorsoften deliberately and systematically evade or ignore the warningsystems, either due to inattention, hurry, misjudgment as to the speedof the train and how quickly the tracks can be crossed, and/or the like.There are numerous cases in which a car or truck has been deliberatelydriven around fully lowered gates and been struck by the train as itattempts to cross the tracks.

Other approaches seek to detect unwanted and dangerous intrusions intothe crossing region and signal alerts to appropriate groups orindividuals. Besides the intruding vehicle/person themselves, otherpossible notification targets are the railroad dispatch office, localfirst responders, and the train operator through an in-locomotivenotification system. These approaches generally include intrusive andnon-intrusive sensor systems. Intrusive sensors require embedment intothe pavement within the crossing. Such devices typically incur periodicfailures due to shifting pavement, freeze/thaw cycles, and vehicleloading effects (e.g., heavy trucks). Sensors in this category includeinductive loop detectors and magnetic sensors, and depend on theinteraction between large metal objects and the electric field producedby the sensor or perturbation of the ambient magnetic field. As such,devices in this class cannot detect humans or any non-metallic objects,and generally cannot detect small metal objects, such as a bicycle or awheelchair.

Non-intrusive sensors operate in a non-contact manner from a distancethat is dependent on the sensor technology, typically measured in feetor tens of feet. Non-intrusive sensors can be further divided intoimaging devices and non-imaging devices. Imaging devices, such as acamera, produce a representation of the object detected in the railroadcrossing, which can be used by a human to further evaluate the nature ofthe object and decide a course of action. Non-imaging devices are merelypresence detectors that give an indication something is present, andpossibly its size. Examples of non-intrusive sensors include: radars(imaging and non-imaging, Doppler and presence sensing); active andpassive acoustic and ultrasonic sensors; laser imaging detection andranging (LIDAR); imaging and non-imaging long wave infrared (LWIR); andvisible spectrum video imaging.

Non-imaging systems tend to be subject to high false alarm rates andpoor spatial resolution. Visible spectrum imaging systems can be subjectto high false alarm rates due to object effects (moving shadows, windeffects, reflections, glint, glare, coloration effects) and do notoperate in fog/smoke/fine drizzle due to the scattering of light by thevery small particles. Radar imaging systems have poor resolution anddifficulty separating objects from background. LWIR systems depend ontemperature differential and are of generally lower resolution thanvisible light systems. These systems operate well in darkness and smokeand fog, but are subject to glare, glint, and background confusion, asare visible spectrum systems. LIDAR (effectively light-based radar)actively scans an area with laser beams, which poses various safetyconcerns and requires an active scanning system, as does radar, whichcan break down.

An ultimate approach is to physically separate the crossing, eitherbuilding a bridge for the train to cross over the road, or a bridge ortunnel for the road to cross over/under the tracks. However, thisapproach is an extremely expensive and time-consuming process, which isnot practical in the vast majority of situations. For example,separating an at-grade crossing in this fashion costs several milliondollars per crossing, and there are over 260,000 grade crossings in theUnited States.

SUMMARY OF THE INVENTION

The inventors recognize several limitations in previous approaches formonitoring an area, such as an at-grade railroad crossing, in whichintrusions into a restricted area in the monitored area can lead toserious consequences. To this extent, the inventors provide a solution,which overcomes one or more of the limitations of these previousapproaches and/or one or more limitations not described herein.

Aspects of the invention provide an assembly for monitoring an area. Theassembly can include two or more cameras sensitive to radiation ofdistinct wavelength ranges. The fields of view of the cameras can besubstantially co-registered at the area to be monitored. The assemblycan include a computer system which can process the image data tomonitor the area. The computer system can be configured to identifyrelevant objects present in the area, update tracking information forthe relevant objects, and evaluate whether an alert condition is presentusing the tracking information.

A first aspect of the invention provides a system comprising: amonitoring assembly deployed to monitor an area, wherein the monitoringassembly includes: a plurality of cameras including at least two camerassensitive to radiation of distinct wavelength ranges, wherein the fieldsof view of the at least two cameras are substantially co-registered atthe area; and a computer system configured to monitor the area byperforming a method including: operating the plurality of cameras tosubstantially concurrently acquire image data of the area; identifying aset of relevant objects in the area using the image data acquired by theplurality of cameras; updating tracking information for each object inthe set of relevant objects; and evaluating an alert condition inresponse to the tracking information for each of the set of relevantobjects.

A second aspect of the invention provides a method comprising: acquiringmultispectral image data of an area on a computer system of a monitoringassembly including a plurality of cameras including at least two camerassensitive to radiation of distinct wavelengths; the computer systemidentifying a set of relevant objects in the area using themultispectral image data; the computer system updating trackinginformation for each object in the set of relevant objects; the computersystem evaluating an alert condition in response to the trackinginformation for each of the set of relevant objects; and the computersystem performing an action response to evaluating a presence of analert condition.

A third aspect of the invention provides an at-grade railroad crossingcomprising: a road intersecting with railroad tracks; and a monitoringassembly deployed to monitor the railroad crossing, wherein themonitoring assembly includes: a first camera sensitive to near infraredand visible radiation; a second camera sensitive to infrared radiation,wherein the fields of view of the first and second cameras aresubstantially co-registered at the intersection of the road and therailroad tracks; and a computer system configured to monitor therailroad crossing by performing a method including: identifying a set ofrelevant objects in the area using image data substantially concurrentlyacquired by the first and second cameras; updating location and trackinginformation for each object in the set of relevant objects; andevaluating an alert condition using the location and trackinginformation for each of the set of relevant objects and an amount oftime until a train next uses the railroad crossing.

Other aspects of the invention provide methods, systems, programproducts, and methods of using and generating each, which include and/orimplement some or all of the actions described herein. The illustrativeaspects of the invention are designed to solve one or more of theproblems herein described and/or one or more other problems notdiscussed.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of the disclosure will be more readilyunderstood from the following detailed description of the variousaspects of the invention taken in conjunction with the accompanyingdrawings that depict various aspects of the invention.

FIG. 1 shows a hypothetical image of an illustrative at-grade railroadcrossing according to an embodiment.

FIG. 2 shows an illustrative system for monitoring operations in an areaaccording to an embodiment.

FIG. 3 shows another illustrative assembly for monitoring operations inan area according to an embodiment.

FIGS. 4A-4D show illustrative examples of multispectral image data,which can be utilized in an embodiment.

FIG. 5 shows an illustrative process for monitoring an area usingmultispectral image data according to an embodiment.

FIG. 6 shows a simulated image of an illustrative at-grade railroadcrossing according to an embodiment.

FIG. 7 shows a hypothetical image of an illustrative entry checkpoint toa secure area according to an embodiment.

It is noted that the drawings may not be to scale. The drawings areintended to depict only typical aspects of the invention, and thereforeshould not be considered as limiting the scope of the invention. In thedrawings, like numbering represents like elements between the drawings.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the invention can provide one or more improvements overprior art approaches for monitoring areas. Illustrative improvementsinclude, for example, multispectral target identification and tracking;multispectral data fusion; calibration and image registration fromenvironmental targets; situation analysis based on multiple imagehistory; and/or the like. In an embodiment, the monitored area includesa restricted area. The restricted area can comprise an area within whichobjects (e.g., people, vehicles, etc.) are not allowed to enter or therestricted area can be an area that can be entered or occupied byvarious objects present in the area at certain times and/or in certaincircumstances.

As indicated above, aspects of the invention provide an assembly formonitoring an area. The assembly can include two or more camerassensitive to radiation of distinct wavelength ranges. The fields of viewof the cameras can be substantially co-registered at the area to bemonitored. The assembly can include a computer system which can processthe image data to monitor the area. The computer system can beconfigured to identify relevant objects present in the area, updatetracking information for the relevant objects, and evaluate whether analert condition is present using the tracking information. As usedherein, unless otherwise noted, the term “set” means one or more (i.e.,at least one) and the phrase “any solution” means any now known or laterdeveloped solution.

Further aspects of the invention are described herein using an at-graderailroad crossing as an illustrative area within which conditions aremonitored. To this extent, FIG. 1 shows a hypothetical image of anillustrative at-grade railroad crossing 10 according to an embodiment.The railroad crossing 10 comprises a location at which a railroad trackor tracks 2 intersect with a road 4 at the same grade (i.e., not aboveon a bridge or below in a tunnel). An illustrative railroad crossing 10includes visual and audible warnings for pedestrians and vehicles on theroad 4. These can include gates 12, lights and bells 14, a crossbucksign 16, and/or the like. Some railroad crossings 10 may only include asubset of these items, such as only the crossbuck sign 16 and lights 14,or even only the crossbuck sign 16.

In any event, various objects may be present in and/or travel throughthe railroad crossing 10. Illustrative objects include: a train 6A, avehicle 6B, a motorcycle 6C, a human 6D, an animal 6E, and debris 6F.When a train 6A is passing through the crossing 10, any other object6B-6F present in the crossing may be struck by the train 6A. To thisextent, a region of the crossing 10 defined by the tracks 2 (and somedistance on either side thereof) becomes a restricted area as a train 6Aapproaches the crossing. To ensure the safety of individuals and petslocated near the crossing as well as the safe passage of the train 6A,it is highly desirable that the restricted area be free of any objects6B-6F. However, as the cost to stop a train 6A, if it is even possible,can be extremely high (five thousand dollars or more), only specifictypes of objects 6B-6F are considered to be of sufficient import to stopthe train 6A prior to its entering the railroad crossing 10. In general,such objects include: humans 6D, as a person is virtually certain to bekilled if struck by a train 6A, it is highly desirable to stop the train6A if at all possible prior to a collision; motor vehicles 6B, 6C, ashumans may be present in such vehicles 6B, 6C and many of such vehicles6B, 6C have sufficient size to potentially cause a derailment if struck;and suspicious or dangerous objects. Suspicious or dangerous objects caninclude objects deliberately placed on the tracks 2, which are generallyassumed to be placed with malicious intent, and accidentally placedobjects, which appear to have a sufficient size and mass to pose aderailment threat should a train 6A collide with the object. To thisextent, objects such as an empty cardboard box 6F, paper, etc., aregenerally ignored, but large pieces of steel fallen from a truck'scargo, or a box apparently deliberately placed on the tracks 2 withunknown contents may be justification for stopping a train 6A orinspection and removal prior to a train 6A using the crossing 10.

As illustrated, the railroad crossing 10 includes a system 20 formonitoring the railroad crossing 10 and the immediate surroundings. Thesystem 20 can be a self-contained system, which includes one or morecameras and a computer system capable of processing the image data to,for example: automatically discriminate between various types of objects6A-6F that may be present in the area; track a history of the movementof objects 6A-6F within the area; make determinations of potentialdanger based on the history, identification of the object(s) 6A-6F,nearness of an approaching train 6A, and/or the like; etc. In responseto determining a potentially dangerous condition is present in therailroad crossing 10, the system 20 can perform one or more actionsincluding, for example: transmitting a warning and/or summary of thesituation to a remote location 8 (e.g., first responders, trainoperations group, and/or the like); transmitting an alert and/orinstructions to an engineer of the train 6A; activating local alertdevices, such as the gates 12, the lights and bells 14, and/or otherwarning devices that can be added to the railroad crossing 10; and/orthe like.

FIG. 2 shows an illustrative system 20 for monitoring operations in anarea, such as a railroad crossing 10 (FIG. 1), according to anembodiment. To this extent, the system 20 includes a computer system 40that can perform a process described herein in order to monitoroperations in the area. In particular, the computer system 40 is shownincluding a monitoring program 50, which makes the computer system 40operable to monitor the area by performing a process described herein.

The computer system 40 is shown including a processing component 42(e.g., one or more processors), a storage component 44 (e.g., a storagehierarchy), an input/output (I/O) component 46 (e.g., one or more I/Ointerfaces and/or devices), and a communications pathway 48. In general,the processing component 42 executes program code, such as themonitoring program 50, which is at least partially fixed in storagecomponent 44. While executing program code, the processing component 42can process data, which can result in reading and/or writing transformeddata from/to the storage component 44 and/or the I/O component 46 forfurther processing. The pathway 48 provides a communications linkbetween each of the components in the computer system 40. The I/Ocomponent 46 can comprise one or more human I/O devices, which enable ahuman user 32 to interact with the computer system 40 and/or one or morecommunications devices to enable a system user, such as a remote system34, to communicate with the computer system 40 using any type ofcommunications link. To this extent, the monitoring program 50 canmanage a set of interfaces (e.g., graphical user interface(s),application program interface, and/or the like) that enable human users32 and/or system users 34 to interact with the monitoring program 50.Furthermore, the monitoring program 50 can manage (e.g., store,retrieve, create, manipulate, organize, present, etc.) the data, such asmonitoring data 54, using any solution.

In any event, the computer system 40 can comprise one or more generalpurpose computing articles of manufacture (e.g., computing devices)capable of executing program code, such as the monitoring program 50,installed thereon. As used herein, it is understood that “program code”means any collection of instructions, in any language, code or notation,that cause a computing device having an information processingcapability to perform a particular action either directly or after anycombination of the following: (a) conversion to another language, codeor notation; (b) reproduction in a different material form; and/or (c)decompression. To this extent, the monitoring program 50 can be embodiedas any combination of system software and/or application software.

Furthermore, the monitoring program 50 can be implemented using a set ofmodules 52. In this case, a module 52 can enable the computer system 40to perform a set of tasks used by the monitoring program 50, and can beseparately developed and/or implemented apart from other portions of themonitoring program 50. As used herein, the term “component” means anyconfiguration of hardware, with or without software, which implementsthe functionality described in conjunction therewith using any solution,while the term “module” means program code that enables a computersystem 40 to implement the actions described in conjunction therewithusing any solution. When fixed in a storage component 44 of a computersystem 40 that includes a processing component 42, a module is asubstantial portion of a component that implements the actions.Regardless, it is understood that two or more components, modules,and/or systems may share some/all of their respective hardware and/orsoftware. Furthermore, it is understood that some of the functionalitydiscussed herein may not be implemented or additional functionality maybe included as part of the computer system 40.

When the computer system 40 comprises multiple computing devices, eachcomputing device can have only a portion of the monitoring program 50fixed thereon (e.g., one or more modules 52). However, it is understoodthat the computer system 40 and the monitoring program 50 are onlyrepresentative of various possible equivalent computer systems that mayperform a process described herein. To this extent, in otherembodiments, the functionality provided by the computer system 40 andthe monitoring program 50 can be at least partially implemented by oneor more computing devices that include any combination of general and/orspecific purpose hardware with or without program code. In eachembodiment, the hardware and program code, if included, can be createdusing standard engineering and programming techniques, respectively.

Regardless, when the computer system 40 includes multiple computingdevices, the computing devices can communicate over any type ofcommunications link. Furthermore, while performing a process describedherein, the computer system 40 can communicate with one or more othercomputer systems, such as the remote system 34, using any type ofcommunications link. In either case, the communications link cancomprise any combination of various types of optical fiber, wired,and/or wireless links; comprise any combination of one or more types ofnetworks; and/or utilize any combination of various types oftransmission techniques and protocols.

In an illustrative embodiment, the computer system 40 receivesmonitoring data 54 from one or more cameras 46A. The camera(s) 46A canbe of any type suitable for operation in the intended operatingenvironment (e.g., a railroad crossing) and have sufficient resolutionand sensitivity to enable the computer system 40 to perform the desiredimage processing as described herein. In an embodiment, the camera(s)46A can include one or more sensing modalities, such as visible light,near-infrared, infrared (e.g., mid-wave infrared, long-wave infrared, orboth, also referred to as thermal infrared), or the like. The computersystem 40 can perform multispectral fusion operations to assist inmaking a determination between normal and abnormal events in the area.Regardless, the computer system 40 can receive monitoring data 54 fromthe camera(s) 46A and process the data to monitor operations in thearea. To this extent, the computer system 40 can implement any of anumber of smart video analysis methods including, for example, thosedescribed in U.S. Pat. Nos. 7,355,508, 8,188,430, and 8,335,606, each ofwhich is hereby incorporated by reference. In an embodiment, thecomputer system 40 is configured to perform a time-based discriminationand event understanding, observing a sequence of images and derive anunderstanding from changes in the view over time, and/or the like.

In some operating environments, there may simultaneously be too littleand too much light within the field of view of a camera 46A. Forexample, during night operations at a railroad crossing 10 (FIG. 1),powerful headlights of locomotives, approaching vehicles, or otherpowerful sources of light may introduce considerable glare into theimage, which can wash out portions of the image, while overall lightingduring the night may be very low. In an embodiment, the computer system40 can operate one or more illuminators 46B to provide sufficient lightto an area being monitored. For example, the computer system 40 candetermine that the area being monitored lacks sufficient light (e.g.,using previously acquired image data, a light sensing device, and/or thelike) and can activate one or more illuminators 46B in response.Similarly, the computer system 40 can determine when the area beingmonitored has sufficient ambient light and can turn off the one or moreilluminators 46B. An illuminator 46B can emit visible or other lightinto the field of view of the camera 46A to provide sufficient lightingfor the camera 46A to acquire image data suitable for processing by thecomputer system 40. In a more particular embodiment, an illuminator 46Bcomprises a near-infrared emitting illuminator (e.g., a light emittingdiode-based illuminator), which can emit acceptable levels ofnear-infrared radiation up to approximately 450 feet (approximately 137meters) for a near-infrared sensitive camera 46A. Furthermore, when twoor more illuminators 46B are used in concert, the distance can readilyexceed 600 feet (183 meters).

To address too much light, e.g., from a locomotive or the like, any ofseveral approaches can be utilized. For example, if the light source isfixed in a portion of the image which is not relevant to the analysis(e.g., distant lights), the portion of the field of view of the camera46A can be physically blacked out. To address situations where the lightsource is too close to a relevant portion of the field of view, thecamera 46A can include a filter, which can be selected to reduce orremove the peak wavelengths of light radiated by the interfering lightsources. Use of such a filter can drastically reduce glare and allow thecomputer system 40 to clearly identify targets in the image dataacquired by the camera 46A, which would otherwise have been obscured.

In an illustrative embodiment, the computer system 40, camera 46A, andilluminator 46B are configured to be deployed together as a singlemonitoring assembly 22. For example, the assembly 22 can comprise asingle enclosure housing all of the components 40, 46A, 46B, or amounting system on which one or more enclosures can be fixed. Thelocation of the assembly 22 can be selected to enable the camera 46A tohave a desired field of view of the area to be monitored.

FIG. 3 shows another illustrative assembly 22A for monitoring operationsin an area according to an embodiment. The assembly 22A can include ahousing holding a pair of cameras 46A1, 46A2, as well as a computersystem 40 and other electronic support components. For example, thecomputer system 40 can comprise a low-power video processing system. Inan embodiment, the computer system 40 is a computer-on-module (COM),such as an Overo COM offered by Gumstix, Inc. The assembly 22A also isshown including an illuminator 46B, which can be operated by thecomputer system 40, when necessary, to provide sufficient light of therequisite spectral bands for the cameras 46A1, 46A2 to acquire imagedata capable of being processed by the computer system 40 up to at leasta required distance. While not shown, it is understood that the housingcan include various other components, which can be operated by thecomputer system 40. For example, the housing can include environmentalcontrols, such as heating and cooling, for a window through which thecameras 46A1, 46A2 acquire image data.

In an embodiment, the cameras 46A1, 46A2 are sensitive to distinctclasses of radiation. For example, a first camera, such as camera 46A1,can be sensitive to visible/near infrared radiation, while the othercamera 46A2 is sensitive to thermal infrared radiation. The two cameras46A1, 46A2 can be mounted and provided with optics such that theirfields of view are substantially co-registered (e.g., greater thanninety-five percent) at a range to the area of interest (e.g., therailroad crossing). In an embodiment, the illuminator 46B comprises anear infrared illuminator (e.g., a near infrared light emittingdiode-based illuminator), which can be operated by the computer system40 to provide near infrared illumination into the area of interest toenable all-lighting operation of the visible/near infrared camera 46A1.

The assembly 22A is shown including various other components. Forexample, the assembly 22A is shown including a transceiver 60 and anantenna 62, which can enable wireless communications between theassembly 22A and a remote system 34 (FIG. 1) and/or a locomotive of atrain 6A (FIG. 1). Such communications can include receiving softwareupdates (e.g., modifications of the target region of interest, newparameters for detection of alerts, and/or the like) from the remotesystem 34, receiving data regarding the monitored area or one or morefuture events that will occur in the monitored area (e.g., an amount oftime before a train will be using a crossing), and/or the like.Furthermore, when the computer system 40 determines a presence of one ormore alert conditions, the computer system 40 can perform one or moreactions, such as communicating data regarding the alert condition to aremote system 34, an approaching train 6A, activate a local alarm (e.g.,a light, a horn, and/or the like) to warn any people that may be indanger, and/or the like. While shown mounted separate from the computersystem 40 and the cameras 46A1, 46A2, it is understood that thetransceiver 60 could be mounted in the same enclosure as the computersystem 40 and/or the cameras 46A1, 46A2. It is understood thatutilization of a wireless communications solution is only illustrativeand any communications solution, including a wired solution, can beutilized.

The assembly 22A can be mounted on a pole 64 of a sufficient height(e.g., approximately forty feet (approximately twelve meters) or higher)to provide a sufficient view of the target area (e.g., the railroadcrossing and the surrounding area) for the cameras 46A1, 46A2. Theassembly 22A can be mounted in a manner that allows the devices (e.g.,the cameras 46A1, 46A2 and/or the illuminator 46B) to be rotated andinclined to an angle appropriate to properly center the fields of viewof the cameras 46A1, 46A2 on the target area. Such a mounting can useany solution and can be permanent or temporary. Furthermore, the variouscomponents can receive power through a wired connection to a power grid.In an embodiment, the assembly 22A includes a local power source, suchas a solar panel 66, which can be configured to provide sufficient powerto enable the operation of the various electronic devices of theassembly 22A without connection to a power grid. In this case, theassembly 22A also can include sufficient power storage (e.g., a battery)to enable operations when the local power source does not generatesufficient power (e.g., no sunlight).

It is understood that various applications may desire to minimize a sizeand/or weight of equipment and enclosures mounted at a top of the pole64. To this extent, an embodiment of the assembly 22A can include onlythe cameras 46A1, 46A2 and illuminator 46B at a high vantage point, withthe remaining components located lower in one or more separateenclosures. In this case, the various components located closer to theground can be more readily serviced and/or upgraded without requiringaccess to a top of the pole 64.

As described herein, the assembly 22A can include multiple cameras 46A1,46A2, which provide multispectral image data of an area being monitored,such as the railroad crossing 10 shown in FIG. 1. FIGS. 4A-4D showillustrative examples of multispectral image data, which can be utilizedin an embodiment. In this case, the image data includes thermal infraredimages 54A, 54C and visible/NIR images 54B, 54D. Images 54A, 54B wereacquired for the same area at substantially the same time (e.g., withinhalf second or less), while images 54C, 54D were similarly acquired forthe same area at substantially the same time.

The computer system 40 (FIG. 2) can receive the images 54A-54D andprocess the images 54A-54D as part of monitoring the area. The computersystem 40 can perform any combination of various types of imageprocessing. For example, the computer system 40 can perform backgroundprocessing and subtraction on each of the images 54A-54D. Illustrativemonitoring data 56A-56D generated after such processing for images54A-54D is shown in conjunction with the respective images 54A-54D. Ascan be seen, each image 54A-54D includes at least one vehicle and personpresent in the imaged area, which are typical targets that may be imagedat the railroad crossing 10.

A significant challenge in much background subtraction processingperformed by a computer system 40 is determining where background andforeground intersect. Typically, many algorithms implemented by thecomputer system 40 can make such a determination based on contrast.While human visual processing can readily identify the car in thevisible/NIR image 54B, making such an identification with the computersystem 40 performing automated image processing is difficult withoutexpending an extremely large amount of available processing. Inparticular, the similar colors and low contrast conspire to make theupper portion of the car effectively disappear in the view of mostanalysis algorithms available for implementation on various types ofcomputer systems 40, which can be located remotely, operate on lowpower, and/or the like.

As most algorithms implemented by a computer system 40 categorizetargets by recognizing shapes in the processed image data 56A-56D, thecomputer system 40 can more readily recognize the imaged car using theprocessed image data 56A as opposed to the processed image data 56B,where the shape remaining after processing is more broken and disjoint.Furthermore, the person in the image data is readily identifiable in theprocessed image data 56A, but is entirely absent from the processedimage data 56B. In light of the above, the infrared image 54A providesimage data more suitable for identifying targets present in the area.However, when considering the images 54C, 54D, which correspond to thesame area, but on a different day, the processed image data 56Dcorresponding to the visible/NIR image 54D enables the computer system40 to identify the vehicle that is present in the images 54C, 54D,whereas the evidence of the vehicle is entirely absent from theprocessed image data 56C for the infrared image 54C.

These results illustrate the differences in functionality provided byuse of multispectral image data. In the first set of images 54A, 54B,the color of the vehicle was close the color of the pavement makingdiscerning the vehicle from the background more difficult for thevisible/NIR image 54B. However, the body of the vehicle was at agenerally lower temperature than the pavement, and therefore could bereadily identified in the processed image data 56A of the infrared image54A. Conversely, hot pavement and a hot vehicle roof present in theinfrared image 54C cause evidence of the vehicle to virtually disappearin the processed image data 56C, while the vehicle remains readilyidentified in the processed image data 56D for the visible/NIR image54D.

The computer system 40 can leverage other differences between the images54A-54D acquired for different spectra. For example, shadows can bedifficult for a computer system 40 to separate from objects, especiallyin the visible light spectrum. However, shadows are often smaller ornonexistent in the infrared spectrum (see, e.g., images 54C, 54D). Whereshadows appear in infrared image data, they are likely due to a vehiclebeing parked for a significant amount of time, after which the shadowcan cause a significant cooling effect on the shaded pavement.Temperature differences also can allow the computer system 40 to use acombination of visible/NIR and infrared to recognize living or motorizedobjects versus nonliving, inert objects. For example, all living thingsin the size range of interest will be at a body temperature usuallysignificantly greater than that of the current environment, and nearlyalways significantly different, especially taking emissivity of variousmaterials into account. Vehicles also generate large amounts of heatwhich will tend to separate them from the background in infrared imagedata.

FIG. 5 shows an illustrative process for monitoring an area usingmultispectral image data, which can be implemented by the computersystem 40 (FIG. 2) in an embodiment. Initially, the computer system 40can obtain image data 54A, 54B for the area, which was acquired bycameras sensitive to unique spectra (e.g., Spectrum A and Spectrum B).For example, the spectra can include the visible/NIR spectrum and thethermal infrared (e.g., long wavelength infrared) spectrum. While thespectra can be non-overlapping, it is understood that the image data54A, 54B can be for partially overlapping spectra. Furthermore, whiletwo spectra are shown and described herein, it is understood that imagedata corresponding to any number of two or more spectra can be utilized.Still further, while infrared and visible/NIR spectra are describedherein, a combination of two or more of any spectra can be utilized.Additional spectra can include for example, visible, ultraviolet,near-infrared, mid-infrared, thermal infrared, subsets or combinationsthereof, and/or the like.

In an embodiment, the image data 54A, 54B is substantially concurrentlyacquired by the cameras. Initially, the computer system 40 canseparately process the image data 54A, 54B, which can be performed bythe computer system 40 in parallel. While the processing actions shownand described for each spectrum of image data 54A, 54B are the same, itis understood that details of the implementation of each processing actby the computer system 40 can be significantly different for image data54A, 54B of different spectra. Such implementations of each of theprocessing actions described herein are known in the art, e.g., asdescribed in the patents, previously incorporated by reference.Furthermore, it is understood that the computer system 40 can implementalternative and/or additional actions to identify target objects withinthe image data 54A, 54B of a spectrum. The process shown and describedherein is only illustrative.

Regardless, in actions 70A, 70B, the computer system 40 can performinitial background modeling on the image data 54A, 54B using anysolution. The initial background modeling can determine what is and isnot background in the image data 54A, 54B. In actions 72A, 72B, thecomputer system 40 can perform background subtraction on the image data54A, 54B to remove all elements from the image data 54A, 54B thatcorrespond to the background. In actions 74A, 74B, the computer system40 can aggregate the remainder of the foreground into blobs (objects)using a blob detection solution. The blob detection solution can beconfigured to reject objects in the foreground, which are below aminimum size as such objects are likely to be spurious artifacts in theimage data. In actions 76A, 76B, the computer system 40 can compare theidentified blobs to previously defined and tracked objects and updateinformation regarding the tracked objects matching a blob accordingly.Furthermore, the computer system 40 can accept or reject objects using aconsistency of movement check. In this case, the computer system 40 candetect and eliminate many spurious objects (e.g., those identified dueto brief shadows or reflections). Similarly, the computer system 40 canassemble multiple “objects” (e.g., due to difficult to identify objectborders) into a single object due to common movement of these objectsover a sufficient period of time and/or distance.

Subsequently, the computer system 40 can confirm the objectsindependently identified in one or more of the image data 54A, 54B. Forexample, in action 80, the computer system 40 can fuse (combine) theobjects found in both of the image data 54A, 54B into a single object,thereby providing a high-confidence, accurate tracking of known objects.The computer system 40 can implement any solution for determining thatobjects are sufficiently similar to correspond to the same object. Forexample, as discussed herein, when the fields of view for the image data54A, 54B are substantially co-registered, the computer system 40 canidentify objects occupying substantially the same location as being thesame object. Furthermore, in actions 78A, 78B, the computer system 40can verify any objects unique to only one spectrum. For example, thecomputer system 40 can perform outline or pattern matching, evaluatewhether the object exhibits consistent movement in the correspondingspectrum image data, and/or the like, to verify that while the object isnot visible in the image data of the other spectrum, it does in factrepresent an actual object and not an illusion of an object.

In action 82, the computer system 40 can add and/or renew all verifiedobjects to a tracked object list. Furthermore, in action 84, thecomputer system 40 can analyze the list of tracked objects in moredetail to identify and/or classify the targets. For example, thecomputer system 40 can take into account any combination of one or morerelevant parameters, such as an overall size of the object, a dimensionratio, speed and vector of movement of the object, and/or the like. Oncethe computer system 40 has classified the tracked objects, in action 86,the computer system 40 can evaluate the overall situation in themonitored area. For example, using the tracking data of an object, thecomputer system 40 can determine whether any object of interest ispresent in a restricted zone (e.g., on or in the immediate vicinity ofthe railroad tracks), how long the object has been present in therestricted zone, whether the object is moving through or out of therestricted zone, and/or the like. Furthermore, the computer system 40can consider other factors, such as whether a critical event will occursoon (e.g., a train is approaching the crossing), weather, trafficpatterns, and/or the like. Depending on the results of the evaluation,the computer system 40 can determine whether an alert condition ispresent in the monitored area, and perform one or more actions inresponse to determining that an alert condition is present.

In various monitoring applications, including the railroad crossingapplication described herein, determination of whether a target is ofinterest includes determining whether a size of the target falls withinan overall size range for relevant targets. For example, small animals,such as rabbits, rats, and/or the like, may be present in a restrictedzone but are not relevant targets for the railroad crossing application.In an embodiment, the computer system 40 can ignore these small animalswhile any human targets are recognized as being relevant. Usingtwo-dimensional image data, determining the actual size of the objectcan depend on knowing a precise distance to the object. In particular,when the precise distance to the object is known, the computer system 40can derive a size of the object from the number of pixels the targetoccupies in the image data 54A, 54B. Without knowing the distance, thenumber of pixels provides little information regarding the size of theobject, which may be very close or very far away.

In an embodiment, the computer system 40 comprises a baseline formeasurement of objects present in the monitored area. For example, thecomputer system 40 can be calibrated using custom targets spaced acrossthe field of view, each having well-known dimensions. The computersystem 40 can process image data acquired with the targets present toperform three-dimensional registration of all of the pixels across thefield of view.

In another embodiment, the computer system 40 can self-calibrate andregister pixels in the image data using available cues (e.g., ambientfeatures) present in the monitored area. To this extent, FIG. 6 shows asimulated image 54 of an illustrative at-grade railroad crossingaccording to an embodiment. For example, the image 54 can correspond toan image acquired by a camera deployed as part of the monitoring system20 at the railroad crossing 10 shown in FIG. 1. The image 54 does notinclude any targets, but does include various static features, such asthe tracks 2, the road 4, the gates 12, the lights 14, the crossbucksign 16, and/or the like.

When present and in use, railroad tracks 2 have highly standardizeddimensions, which are maintained to considerable precision in order toallow for the safe travel of trains. In general, tracks 2 include atleast two rails 2A, 2B, which are supported by a series of ties 2C andare fastened such that they are located a fixed distance 3 apart. In theUnited States, this distance 3A is generally four feet, eight andone-half inches (approximately 1.44 meters), and is maintained to avariation of less than approximately 0.5 inches (approximately 1.3centimeters). Should the rails 2A, 2B have a higher variance, a dangerof train derailment due to a mismatch of rail gauge and wheel-to-wheeldimensions on the train increases. Similarly, the ties 2C typically havestandard sizes and are spaced at a fixed distance from one another. Forexample, in the United States, a typical tie 2C measures approximately8.5 feet long by nine inches wide (approximately 2.6 meters by 22.9centimeters), has a thickness of approximately seven inches(approximately 17.8 centimeters) and is spaced from adjacent ties byapproximately twenty-one inches (approximately 53.3 centimeters). Thetypical geometry of the various components 2A-2C of the track 2 meansthat the ties 2C are at right angles to the rails 2A, 2B.

The rail components 2A-2C visible in the image 54 provide the computersystem 40 with multiple geometric references with which to create anaccurate registration map (e.g., a two-dimensional projection of athree-dimensional model) of the imaged field of view. For example, thecomputer system 40 can use the distance between the visible tops of therails 2A, 2B in the image 54 and the known separation of the rails 2A,2B to create the registration map. Furthermore, the computer system 40can use the ties 2C as guidelines to generate a grid of parallel lines58A, 58B, which will be separate from each other by the center-to-centerspacing distance 3B of the ties 2C. Furthermore, the computer system 40can use other non-rail features, which may have known measurements, suchas the crossbuck sign 16, the road 4, the gates 12, the lights 14,and/or the like, to create the registration map. In an embodiment, whendeployed, the mounting height of the camera 46A (FIG. 2) of themonitoring system 20 is measured and a horizontal distance from thecamera 46A to a visible location in the area 10, such as a selected tie2C is measured. Furthermore, measurement data for one or more staticfeatures present in the image 54 (e.g., actual spacing between two ormore ties 2C, markings on the road 4, and/or the like) can be obtained.Using these measurements and the known geometric references provided bythe track components 2A-2C, the computer system 40 can derive a detailedand accurate assessment of the distances to, and visibility of, keylocations throughout the field of view. However, after calibration,these landmarks no longer need to be visible within the image data(e.g., can be removed, covered by snow, etc.) as their locations havebeen virtualized in the registration map. With such knowledge, thecomputer system 40 can readily determine the three-dimensionalequivalence of various locations in the two-dimensional image data withor without various landmarks being visible in the image data.

Furthermore, while monitoring the area 10 and one or more objectspresent in the area 10, the computer system 40 can match a location(e.g., a location at which the object touches the ground) of the objectto the key locations and can determine precise coordinates of the objectusing the registration map. Using the coordinates and pixel extent inthe image 54, the computer system 40 can determine a size andorientation of the object. Furthermore, the computer system 40 can useadditional information to verify location and orientation of an object.To this extent, the computer system 40 can analyze a relationship of theobject with one or more static objects present in the field of view. Forexample, the vehicle 6B shown in FIG. 1 is partially blocking a view ofat least one of the rails of the tracks 2. As a result, the computersystem 40 can determine that the vehicle 6B has not yet passed bothrails of the tracks 2. Similarly, the computer system 40 can use otherpermanent objects or features in the field of view of a monitored area10 to provide additional calibration and position references.

As described herein, any target object's distance can be determined bythe point or points on the ground plane (e.g., the road 4, the tracks 2,and/or the like) at which the object touches and projecting upward intothe calibrated three-dimensional space of the registration mappreviously calculated by the computer system 40. It is understood thatvarious objects, such as blowing debris, birds, and/or the like, may bepresent within the image 54, but not be located on the ground. In anembodiment, the computer system 40 can eliminate such objects fromconsideration when processing the image data, e.g., due tocharacteristic movement of the object, the relationship of the object toother objects in the field of view, and/or the like.

In a monitoring application, such as the railroad crossing applicationdescribed herein, a basic function of the monitoring system 20 is toidentify when an alert condition (e.g., a potentially dangerouscircumstance) exists within the monitored area 10. However, for manyapplications, it is equally important for the computer system 40 toaccurately determine when an alert condition does not exist so thatunnecessary actions are not taken in response to circumstances that didnot merit a response. As described herein, the computer system 40 canaccurately determine whether an object is present in a restricted area(e.g., within the grade crossing region) or outside of the area anddetermine an alert condition accordingly.

In addition, as described herein, the computer system 40 can beconfigured to discriminate between objects (e.g., a vehicle, a human,and/or the like) present in the restricted area which correspond to analert condition and warrant actions being taken in response thereto, andobjects (e.g., small wildlife) which do not correspond to an alertcondition and do not require any action(s). In an embodiment, thecomputer system 40 can use the size and shape of the object and trackedbehavior of the object (e.g., walking) to classify the object as beingrelevant to an alert condition or not relevant to an alert condition.

Furthermore, the computer system 40 can use information provided by theuse of multispectral image data to further assist in classifying theobject. For example, near-infrared image data can be obtained in verylow-light conditions and can be enhanced using one or more near infraredilluminators. The use of near infrared illumination is invisible to thehuman eye, and therefore does not provide a distraction or glare sourcefor drivers. Furthermore, near infrared imagery can penetrate some formsof obscuration significantly better than visible light imagery.Additionally, thermal infrared imagery does not require illumination,and can penetrate smoke, fog, precipitation, and/or the like. Thermalinfrared image data also provides information regarding temperaturedifferences between different objects in the image data. The computersystem 40 can use the temperature data to distinguish between operatingvehicles and living creatures, which generate significant amounts ofheat, and other natural objects, which generally do not generate heat.Using a combination of image data from different spectra, the computersystem 40 can accurately discriminate between likely types of targetobjects, while ensuring an ability to monitor the entire area 10 and allpotential target objects regardless of time of day, current weatherconditions, and/or the like.

Furthermore, the computer system 40 can use tracking data for an objectto discriminate between objects which have similar appearances butdifferent threat levels. For example, using the box 6F shown in FIG. 1as an example, using tracking data for the box 6F, the computer system40 can determine that the box 6F is blowing in the wind and thereforenot a threat to the approaching train 6A (FIG. 1). Additionally, thecomputer system 40 can determine that the box 6F is open and empty andtherefore not a threat. However, when the box 6F is closed and thetracking data indicates the box 6F has been located on the tracks 2without moving, the computer system 40 cannot determine whether the box6F constitutes a true threat or not. For example, the box 6F could stillbe empty, but the box 6F also could be filled with metal, explosives,and/or the like, which will present a threat to the train 6A and otherobjects 6B-6E in the area 10.

To address the latter situation and/or other similar situations, thecomputer system 40 can use the tracking data for the object 6F to make ajudgment regarding the object 6F using any approach, which can be simpleor complex to implement. For example, the computer system 40 can trackobjects moving into and out of the restricted area (e.g., where thetracks 2 cross the road 4) and determine whether an object has remainedin the restricted area longer than expected. Using the vehicle 6B(FIG. 1) as an illustrative object, the computer system 40 can track themovement of the vehicle 6B from the time it enters the field of view asit approaches the crossing, slows down, and begins to cross. The speedof the vehicle 6B can be determined by inter-frame movement of thevehicle and use of the registration map described herein. In the eventthat the speed of the vehicle 6B drops to zero and the vehicle 6Bremains in the crossing region for more than a short period of time, thecomputer system 40 can determine that the vehicle 6B has encounteredsome problem (e.g., mechanical, stuck, and/or the like). Depending on anamount of time to a critical event (e.g., a location of the nearesttrain 6A and an amount of time before it uses the railroad crossing) andthe length of time that the vehicle 6B remains in the restricted area,the computer system 40 can determine that an alert condition exists andinitiate action in response thereto.

In the case of the box 6F, the computer system 40 can use the trackingdata to determine that the box 6F entered the field of view withoutaction by another object, has been moving in a manner characteristic oftumbling (e.g., periodically leaving the ground plane and/or tumblingquickly in the wind), and/or the like. To facilitate a judgment as towhether wind-assisted movement is expected for the box 6F, the computersystem 40 can use data acquired from other components, such as a windspeed sensor (anemometer), which can be deployed as part of themonitoring assembly 22 (FIG. 2).

For a box 6F containing heavier material, such a box 6F would not beexpected to move in the wind and would have been placed, eitheraccidently (e.g., falling from a vehicle) or purposefully, in therestricted area by another object. To this extent, the computer system40 can use tracking data for other objects and knowledge regarding theconsistency of shapes and objects to determine how the box 6F arrived atthe location. For example, if a vehicle 6B enters and leaves therestricted area and the box 6F is first identified as being locatedwithin the restricted area after such time, the computer system 40 candetermine that the box 6F was left in the restricted area by the vehicle6B.

Furthermore, the computer system 40 can determine the manner in whichthe box 6F was placed in the restricted area. For example, if the box 6Ffalls from the vehicle 6B and the fall is captured in the image data,the computer system 40 can analyze the fall to determine an estimate ofa density of the box 6F as an empty/light box will tend to falldifferently than a box of the same size loaded with heavy materials. Thecomputer system 40 can infer intent from the tracked movement of thevehicle 6B. For example, if the vehicle 6B is seen temporarily stoppingin the restricted area, or going over a bump at a high rate of speed,the computer system 40 can infer that the box 6F was purposefully or notpurposefully placed in the restricted area. Similarly, if the computersystem 40 tracks the box 6F to one or more humans 6D (FIG. 1)purposefully placing the box 6F in the restricted area, the computersystem 40 can infer malicious intent and a potential danger to anapproaching train 6A. It is understood that these situations are onlyillustrative of various similar circumstances that the computer system40 can recognize using any known solution of analysis or processing.

In some applications, an entirety of the restricted area is not visibleto the camera(s) of the monitoring system 20. For example, arailroad-related restricted area extends along an entirety of therailroad tracks 2, not just at the railroad crossing 10 and thesurrounding area. To this extent, the computer system 40 can determinean alert condition when an object of interest is within the restrictedarea when it moves out of the field of view of the camera(s). In thiscase, the monitoring system 20 can perform one or more actions, such asnotifying some relevant personnel, providing information to anothermonitoring system 20, which can commence tracking the object, and/or thelike.

While aspects of the invention have been shown and described inconjunction with a solution for monitoring an at-grade railroadcrossing, it is understood that embodiments of the invention can beimplemented to monitor any of various types of areas for any of variousreasons.

For example, an embodiment can be utilized to monitor an area forsecurity purposes, such as a portion of a secure area. To this extent,security for various installations, including private and militaryinstallations, often relies on human observers who are present at anentrance or in remote observation areas. In either case, humans can bedistracted or inattentive while providing security. Furthermore,maintaining a significant manned presence at a checkpoint or in anobservation area also can be a significant ongoing expense. Stillfurther, humans have a difficult time seeing in various ambientconditions, including rain, darkness, snow, etc.

FIG. 7 shows a hypothetical image of an illustrative entry checkpoint toa secure area 110 according to an embodiment. As illustrated, amonitoring system 20 can be located just outside the secure area and bemounted such that a field of view of the camera(s) of the system 20includes a fence 106A, a gate region 106B, and a checkpoint building106C of the entry checkpoint. Furthermore, a road 104 is visible in thefield of view, on which various vehicles 108 may approach seeking toenter the secure area 110. As illustrated, the monitoring system 20 canbe located on an opposite side of the road 104 as the checkpointbuilding 106C, where a guard will be normally located.

As described herein, the monitoring system 20 can be configured toconstruct a three-dimensional representation of the entry checkpointarea. To this extent, the fence 106A (e.g., with evenly spaced poles ofknown heights and/or evenly spaced wiring), the gate region 106B (e.g.,having a known width), the checkpoint building 106C (e.g., havingdimensions which are known or readily measured), and road 104 (e.g.,including markings and/or known dimensions) provide references, whichcan enable the computer system 40 (FIG. 2) of the monitoring system 20to self-calibrate and register pixels in the image data using anapproach similar to that described in conjunction with the railroadcrossing. During operation, as a target, such as a vehicle 108,approaches the gate region 106B, the computer system 40 can monitor itsbehavior for an alert condition. For example, in the event that allvehicles 108 are required to stop prior to being allowed to pass throughthe gate region 106B, the computer system 40 can identify an alertcondition and immediately initiate action in response thereto should thevehicle 108 not stop.

In an embodiment, the field of view of the camera(s) of the monitoringsystem 20 covers a significantly larger area than the road 104 and thegate region 106B. In this case, the monitoring system 20 can detectattempts by individuals to scale the fence 106A even as the guard'sattention at the checkpoint building 106C is diverted. To this extent,additional monitoring systems 20 could be deployed to provide fullcoverage of a perimeter of the secure area. Furthermore, a personattempting to sneak through the gate region 106B using a far side of thevehicle 108 from the guard as cover can be detected by the monitoringsystem 20, which can initiate appropriate action in response. Stillfurther, the monitoring system 20 can identify particular anomalouscharacteristics of a vehicle 108, such as, for example, an anomalousheat source when the camera(s) include a thermal infrared camera. Use ofmultispectral imaging can make evading detection when entering thesecure area 110 extraordinarily difficult.

In another application, a monitoring system 20 can be utilized as anon-demand monitor, such as a traffic monitor. Current traffic monitorapproaches have limited processing capabilities and perform most oftheir computation in a central location. In contrast, a monitoringsystem 20 described herein can perform all analysis and decision-makingonboard and can offer significantly greater performance than otherapproaches. For example, an embodiment of the monitoring system 20 canbe configured to perform complex analysis, such as for planning andother public works purposes, which can provide important details fromthe behavior and volume of both vehicular and pedestrian traffic. Suchanalysis can include, for example: particular paths of travel that aremost common for pedestrians as well as vehicles, time of day when volumetypically peaks or is minimal, locations of bottlenecks for traffictraveling through an area, general flow of traffic through variousintersections (e.g., amount of stopping/starting due to mistimed trafficsignals), and/or the like. Furthermore, the monitoring system 20 can beinstalled on any convenient pole overlooking a region of interest, suchas an intersection, and can perform self-calibration as described hereinusing ambient features, such as the road, and can be ready for usewithin a few minutes of installation. Moreover, as there are no specificgeneral geometric requirements, a monitoring system 20 can be used andre-deployed at virtually any intersection, roadway, bridge, or otherarea of infrastructure for which monitoring is desired.

Still another embodiment can be utilized to monitor a sensitive area forsecurity purposes. For example, railroads have an interest in preventingtrespass into areas near and inside railroad tunnels and other safety orsecurity sensitive areas. Not only is the likelihood of any intruderbeing injured or killed once inside a tunnel very high, but alsoanything which causes an accident within a tunnel has a much higherpotential of serious damage and expense as it is considerably harder toclear a tunnel than an open section of track. In this embodiment, amonitoring system can be located such that an entire region near thesensitive area (e.g., a tunnel, tracks entering the tunnel, and otherobjects in the vicinity) are within the field of view. The computersystem 40 (FIG. 2) can be configured to use objects in the field ofview, such as the tunnel and tracks, and possibly other objects in thevicinity, as references for deriving the three-dimensional imagingspace. The computer system 40 can perform similar analyses of behaviorand objects in order to detect unwanted intrusions, identify suspiciousobjects, alert appropriate personnel or organizations, and/or the like,which can be tailored to the particular requirements of the application.

While shown and described herein as a method and system for monitoringan area, it is understood that aspects of the invention further providevarious alternative embodiments. For example, in one embodiment, theinvention provides a computer program fixed in at least onecomputer-readable medium, which when executed, enables a computer systemto monitor an area. To this extent, the computer-readable mediumincludes program code, such as the monitoring program 50 (FIG. 2), whichenables a computer system to implement some or all of a processdescribed herein. It is understood that the term “computer-readablemedium” comprises one or more of any type of tangible medium ofexpression, now known or later developed, from which a copy of theprogram code can be perceived, reproduced, or otherwise communicated bya computing device. For example, the computer-readable medium cancomprise: one or more portable storage articles of manufacture; one ormore memory/storage components of a computing device; paper; and/or thelike.

In another embodiment, the invention provides a method of providing acopy of program code, such as the monitoring program 50 (FIG. 2), whichenables a computer system to implement some or all of a processdescribed herein. In this case, a computer system can process a copy ofthe program code to generate and transmit, for reception at a second,distinct location, a set of data signals that has one or more of itscharacteristics set and/or changed in such a manner as to encode a copyof the program code in the set of data signals. Similarly, an embodimentof the invention provides a method of acquiring a copy of the programcode, which includes a computer system receiving the set of data signalsdescribed herein, and translating the set of data signals into a copy ofthe computer program fixed in at least one computer-readable medium. Ineither case, the set of data signals can be transmitted/received usingany type of communications link.

In still another embodiment, the invention provides a method ofgenerating a system for monitoring an area. In this case, the generatingcan include configuring a computer system, such as the computer system40 (FIG. 2), to implement the method of monitoring an area. Theconfiguring can include obtaining (e.g., creating, maintaining,purchasing, modifying, using, making available, etc.) one or morehardware components, with or without one or more software modules, andsetting up the components and/or modules to implement a processdescribed herein. To this extent, the configuring can include deployingone or more components to the computer system, which can comprise one ormore of: (1) installing program code on a computing device; (2) addingone or more computing and/or I/O devices to the computer system; (3)incorporating and/or modifying the computer system to enable it toperform a process described herein; and/or the like.

The foregoing description of various aspects of the invention has beenpresented for purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formdisclosed, and obviously, many modifications and variations arepossible. Such modifications and variations that may be apparent to anindividual in the art are included within the scope of the invention asdefined by the accompanying claims.

What is claimed is:
 1. A system comprising: a monitoring assemblyconfigured to monitor an area, wherein the monitoring assembly includes:a computer system including means for monitoring the area, wherein themeans for monitoring is configured to: identify a set of relevantobjects in the area using image data of the area, wherein theidentifying includes evaluating a size of an object, a shape of theobject, and a consistency of movement of the object, to classify theobject as either a relevant object associated with at least one alertcondition or an irrelevant object not associated with any alertcondition; update tracking information for each object in the set ofrelevant objects, wherein the tracking information includes a history ofmovement in the area of each object in the set of relevant objects; andevaluate an alert condition using the updated tracking information foreach of the set of relevant objects, wherein the evaluating includesevaluating a presence of a relevant object in a restricted zone withrespect to the alert condition using tracking information regarding: anamount of time the relevant object has been in the restricted zone, amanner in which the relevant object entered the restricted zone, and amanner in which the relevant object has been moving within the area. 2.The system of claim 1, wherein the monitoring assembly further includesa plurality of cameras including at least two cameras sensitive toradiation of distinct wavelength ranges, wherein the fields of view ofthe at least two cameras are substantially co-registered at the area,wherein the image data includes image data substantially concurrentlyacquired by the at least two cameras, and wherein the identifyingincludes: independently processing the image data acquired by each ofthe plurality of cameras to identify objects present in the area; fusingsimilar objects identified in the image data acquired by each of theplurality of cameras; and verifying objects identified in the image dataacquired by only a portion of the plurality of cameras.
 3. The system ofclaim 2, wherein the plurality of distinct wavelength ranges includes:near infrared and visible radiation and infrared radiation, themonitoring assembly further comprising a near infrared illuminator,wherein the means for monitoring is further configured to operate thenear infrared illuminator to provide sufficient light for imaging thearea with the first camera.
 4. The system of claim 1, wherein theidentifying a set of relevant objects further includes classifying anobject identified in the area as being relevant or irrelevant based onan orientation of the object.
 5. The system of claim 1, wherein theidentifying a set of relevant objects further includes classifying anobject identified in the area as being relevant or irrelevant based on atemperature difference between the object and at least one other objectin the image data.
 6. The system of claim 1, wherein the computer systemis further configured to self-calibrate using a set of static featuresvisible in the image data of the area.
 7. The system of claim 6, whereinthe set of static features includes at least one of: rails or ties forrailroad tracks.
 8. The system of claim 1, wherein the evaluating analert condition includes discriminating between possible objects havingsimilar appearance but different threat levels using at least one of:tracking data for the object in the monitored area or tracking data foranother object associated with the object in the restricted area.
 9. Thesystem of claim 8, wherein the evaluating further uses an amount of timeto a critical event.
 10. The system of claim 9, wherein the restrictedarea comprises a restricted path periodically used by authorizedvehicles and the critical event comprises an authorized vehicle movingthrough the restricted area using the restricted path.
 11. A methodcomprising: a computer system identifying a set of relevant objects inan area using image data of the area, and wherein the identifyingincludes evaluating a size of an object, a shape of the object, and aconsistency of movement of the object, to classify the object as eithera relevant object associated with at least one alert condition or anirrelevant object not associated with any alert condition; the computersystem updating tracking information for each object in the set ofrelevant objects, wherein the tracking information includes a history ofmovement in the area of each object in the set of relevant objects; thecomputer system evaluating an alert condition using the updated trackinginformation for each of the set of relevant objects, wherein theevaluating includes evaluating a presence of a relevant object in arestricted zone with respect to the alert condition using trackinginformation regarding: an amount of time the relevant object has been inthe restricted zone, a manner in which the relevant object entered therestricted zone, and a manner in which the relevant object has beenmoving within the area; and the computer system performing an actionresponse to evaluating a presence of an alert condition.
 12. The methodof claim 11, further comprising the computer system self-calibrating andregistering two-dimensional pixel data of the area into athree-dimensional model of the area using a plurality of static featuresvisible in the image data of the area.
 13. The method of claim 12,wherein the plurality of static features include at least one of: railsor ties for railroad tracks in the area.
 14. The method of claim 11,wherein the identifying a set of relevant objects further includesclassifying an object identified in the area as being relevant orirrelevant based on an orientation of the object.
 15. The method ofclaim 11, wherein the identifying a set of relevant objects furtherincludes classifying an object identified in the area as being relevantor irrelevant based on a temperature difference between the object andat least one other object in the image data.
 16. The method of claim 11,wherein the evaluating an alert condition includes discriminatingbetween possible objects having similar appearance but different threatlevels using at least one of: tracking data for the object in themonitored area or tracking data for another object associated with theobject in the restricted area.
 17. A system for monitoring an areaincluding a restricted area, the system comprising: a first camerasensitive to near infrared and visible radiation; a second camerasensitive to infrared radiation, wherein the fields of view of the firstand second cameras are substantially co-registered at the area; and acomputer system including means for monitoring the area, where in themeans for monitoring is configured to: identify a set of relevantobjects in the area using image data substantially concurrently acquiredby the first and second cameras, wherein the identifying includesevaluating a size of an object, a shape of the object, and a consistencyof movement of the object, to classify the object as either a relevantobject associated with at least one alert condition or an irrelevantobject not associated with any alert condition; update location andtracking information for each object in the set of relevant objects,wherein the tracking information includes a history of movement in thearea of each object in the set of relevant objects; and evaluate analert condition using the location and tracking information for each ofthe set of relevant objects and an amount of time until a criticalevent, wherein the evaluating includes evaluating a presence of arelevant object in a restricted zone with respect to the alert conditionusing tracking information regarding: an amount of time the relevantobject has been in the restricted zone, a manner in which the relevantobject entered the restricted zone, and a manner in which the relevantobject has been moving within the area.
 18. The crossing of claim 17,wherein the updating uses a registration map mapping the image data to athree-dimensional model of the railroad crossing.
 19. The crossing ofclaim 17, wherein the identifying a set of relevant objects furtherincludes: identifying all objects visible in the image data; determininga size and orientation of each object visible in the image data; andclassifying an object as relevant or irrelevant based on at least oneof: the size and orientation of the object or tracking informationcorresponding to movement of the object.
 20. The crossing of claim 17,the system further comprising a near infrared illuminator, wherein themeans for monitoring is further configured to operate the near infraredilluminator to provide sufficient light for imaging the restricted areawith the first camera.